Conjugate Gradient Algorithm for Optimization Under Unitary Matrix Constraint1

In this paper we introduce a Riemannian algorithm for minimizing (or maximizing) a real-valued function J of complex-valued matrix argument W under the constraint that W is an n × n unitary matrix. This type of constrained optimization problem arises in many array and multi-channel signal processing applications. We propose a conjugate gradient (CG) algorithm on the Lie group of unitary matrices U(n). The algorithm fully exploits the group properties in order to reduce the computational cost. Two novel geodesic search methods exploiting the almost periodic nature of the cost function along geodesics on U(n) are introduced. We demonstrate the performance of the proposed conjugate gradient algorithm in a blind signal separation application. Computer simulations show that the proposed algorithm outperforms other existing algorithms in terms of convergence speed and computational complexity.